Implementation and evaluation of a German HMM for POS 

disambiguation 



in 

On 
On 

(N 



> 

oo 
m 
O 

CM 
O 

in 



X 



Helmut Feldweg 

Department of Linguistics 
University of Tubingen 
Wilhelmstr. 113 
72074 Tubingen 
Germany 
Helmut . Feldweg@uni-tuebingen.de 



Abstract 



A German language model for the Xerox 
HMM tagger is presented. This model's 
performance is compared with two other 
German taggers with partial parameter re- 
estimation and full adaption of parameters 
from pre-tagged corpora. The ambiguity 
types resolved by this model are analysed 
and compared to ambiguity types of En- 
glish and French. Finally, the model's error 
types are described. I argue that although 
the overall performance of these models for 
German is comparable to results for En- 
glish and French, a more exact analysis 
demonstrates important differences in the 
types of disambiguation involved for Ger- 



1 Background 

Since the late '80s part-of-speech (POS) disam- 
biguation using Hidden Markov Models (HMM) has 
been a widespread method for tagging texts. Despite 
this fact, little work has been done so far toward 
employing this technology for the disambiguation of 
German texts (cf. ( Wothke et al., 1993 ), ( Schmid 
and Kempe, 1995|) ) . Earlier work of the author 
(Feldweg 1993 and 1995) within the framework of 
a project on corpus based development of lexical 
knowledge bases (ELWIS) has produced LIKELY, 
a straightforward implementation of the Viterbi al- 
gorithm employing an HMM whose parameters were 
obtained from a pre-tagged text corpus. Since then 
the original tag set was redefined, making the tagged 
corpus used to train the LIKELY tagger obsolete. 

Within a current project on adapting bilingual 
dictionaries for online comprehension assistance 
(COMPASS, LRE 62-080), the need arose for a 
POS-disambiguator to facilitate a context sensi- 
tive dictionary look-up system. As the COMPASS 



project makes ample use of Xerox technology for its 
core look-up engine and for POS disambiguation for 
languages other than German, the obvious thing to 
do was to develop a German language model for the 
Xerox tagger. 

The following section describes the implementa- 
tion of this new model. In section |^ the results ob- 
tained using the new model are compared with the 
results from previous models. An analysis of the 
types of disambiguation involved in these models is 
presented in section |4|. The model's error types are 
analysed in section |5|, and conclusions are drawn in 
section [| 

2 Implementation of the German 
language model 

The version of the Xerox tagger used for the imple- 
mcn tation described here is the DDS Tagger version 
1 . 1 ( Kupicc and Wilkcns, 1994 ) . This version differs 
from the curr ent version (1.2) of th e Xerox Tagger as 
described in QCutting et al., 1992| ) in that the DDS 
Tagger accommodates lexicons and class guessers in 
the form of external finite-state transducers. Imple- 
menting a new language model for this tagger in- 
volves supplying: 

(1) a definition of the tag set to be used by the 
HMM, 

(2) a lexicon listing word forms with their equiva- 
lence classes, that is the list of POS tags that 
can be assigned to the word form, 

(3) a class guesser that assigns equivalence classes 
to words not covered by the lexicon, 

(4) a set of initial transition biases, 

(5) a set of initial symbol biases, 

(6) a sufficiently large text for training the HMM, 



(7) a reference text with correctly assigned POS 
tags, 

(8) a tokenizer that recognizes words in free text. 

The following paragraphs describe these compo- 
nents in more detail. 

(1) The tag set used in the implementation 
is the smaller version of the two tag sets de- 
veloped jointly by the Universities of Stuttgart 
and Tubingen, referred to as the ELWIS tag set 
(cf. ( |Thielen, 1994| ) and QThiclcn and Schiller, 
1995| )). It consists of a total of 42 POS tags plus 
three tags for punctuation and a special tag for trun- 
cated words. The tags used in the ELWIS tag set 
are given in table [|. 

(2) For each word form the lexicon gives the 
set of POS tags that can be assigned to that word. 
This set may consist of one (for unambiguous words) 
or more tags and is referred to as the word's equiva- 
lence class. The lexicon used in this implementation 
was derived from Lingsoft's GERTWOL morpholog- 
ical analyzer, which uses finite state transducers, by 
mapping the morpho-syntactic labels generated by 
GERTWOL onto the ELWIS tag set. The lexicon 
is realized as a finite state transducer for the DDS 
Tagger. Alternative mapping rules were developed 
to generate a lexicon with ELWIS tags from the Ger- 
man lexical database of the Centre for Lexical Infor- 
mation ( pELEX, 1993D . 

(3) Class guessers for the Xerox tagger assign 
potential POS tags to unknown words according 
to a surface analysis of the word form. In addi- 
tion to the common practice of mapping POS tags 
according to the words' suffixes, this implementa- 
tion makes use of the case of the initial letter of a 
word — which is highly significant for POS assign- 
ment in German. The class guesser also takes care 
of POS-assignmcnt for abbreviations, special sym- 
bol sequences and language external material in the 
text. The class guesser, like the lexicon, is a finite 
state transducer. 

(4) The model is trained using a set of ini- 
tial transition biases, including both positive and 
negative constraints on tag sequences. Although 
the model can be trained without initial biases, the 
performance of the resulting model increases signif- 
icantly if appropriate initial biases are used. 

The biases in the model consisted for the most 
part in specifications of the most plausible successor 
tags for each tag in the tag set. They were con- 
structed manually and refined in a series of subse- 
quent training and evaluation runs. 



Label 


Part— of— Sneech 


MM 
IN 1 N 


noun 


NE 


ataapt noun 

pi Upul Ll\J Llll 


VFIN 


frnitp vprn 

lllllLL VCl u 


VINF 


infinitive verb 


VIZU 


irmmtivp vprn with 711 

XJ.llli.lJ. Ll VVj V Lii l-J VV J Lil Aj LL 


VPP 
v r r 


JJcLDL JJcLl LluipiC 


ART 


article 


ADJA 


a n ippti vp a ttri ni 1 1 1 vp 

ClV_l 1 Ui V U , Clr U Ll L JL V U 


ADJD 


arlipetivp aHvpr l~>ial 


PPER 


npTQnnfl atati Ann it'T'Otipyr/'P 

pul OUlldl JJl U11U llll ^ 111 C11CA1 V G 


PPPRPF 
I I I . U I u 


pel DUllcll pi UI1UU.I1, 1 UlluXl V(J elilU 




irreflexive 


PRF 


iirnn Al 1 n rprl piti vp 

JJl U11U Llll , 1 CU-lCiAJ. V U 


i jr woo 


JJl U11U Llll, pUttoUool VL, o UUDLelilLl Vc 


PPOS AT 


AT An Al 1 Ti A AQQPQQ1 VP P\ TTT1 Al 1 T 1 A Tl Pi 
JJl U11U Llll, pUoSLBSIVL, ClL Ll 1U LlLlUlldl 




pi UllULlll, ULlllUllb Ll clLl Vt- ctllU 




111UU11111 LL- 


PROAT 


AT An ATTT infl PI H VPT A 
JJl UllUllllllCll CLL1 V t_.l LJ 


PWS 


TIT ATI Al 1T1 lTltpTTAlXPlTl VP 

JJl U11U Ull , 111LU1 1 U JiCl Ll VCj 




Qll licit" 9\ Tltl VP 
O Li UD LCL11L1 V C- 


PWAT 


nrnnnnn rntprrACPi ti vp 

J_/l U11U Llll , 111 L v^J 1 U^Cl LIVL^ 




flftTl All 1 1 VP 
CLLLIILJLLLIVC 


PRELS 

_1_ _L I I 1 J_J kJ 


ATATTA11T1 TplfltlVP till AQt fl Tl 1 1 VP 

JJl U11U Llll , 1 C1C1 u LIUu LdllLl V 


PRTlT at 

JT lAj J_J T.J -t\ 1 


T~*iT"m ai i n i" o ofn a r rn r\n fnro 
JJl U11U Llll, 1 clclLl V c, cLLLllUULlVC 


PNFT 


nt"An Ann tiati mHfipunn' 
JJl UllULlll, 11U11 lllllUuLlllji 


PALL 


TIT ATI Al 1 Tl TATTTT Q AT nil — 
JJl U11U Llll , 1UI lllO Ul Lit t 


PRFTn 


nrnnnnn tya vm g at hoij// 

JJl U11U Llll , 1U1 lllo Ul UC- IUj 


PVIEL 


TT ATI 1 TTTI PPT"in O" Tl ATI P\ 1 1"T1 Al it" 1 A^P 
11U11 11111CU Llllg, , 11U11 ClLLl 1UUL1 VC 




TATTTm AT 1)1 Pi — IDPHIfi pt.P 

IvJl lllO Ul » LU O / V u\J ^lJV« 


CARD 


numbers, Ccirdincil 


ADV 


Reiver bs 


KOUI 


pnnin Tipti ati mmnvni M/rtinp" wit n 

VjVJllJ LLIIVj 11U11 « OUUUl V1111CL llllt VV 1 111 




in nil i ti vp PATTrnlpti ati 

11111111 Ll V U LU111 U1L L 1U11 


KOUS 


conjunction, subordinating with. 




finite completion 


KON 


o nnnmptin n pa at a i ti pi rm o' 

LjUIIJ LL11L L1U11, LUUI Lllllcl Llllt 


KOKOM 

_i_ yvj _i_ yvj iv_l 


p A numf'ti A n p atti api tpi ti vp 

LjUII J LlllL L1U11, UUllipdl CX Ll V C 


APPR 


ni'PnnQi fmn 
pi CpUol L1U11 


APPO 


pUD LpUolLlUll 


APZR 


P1TP11TT1 A A^l 1 1 ATI TlCnt APITt 
Ull U LllllJJ'Jol L1LJ11, 11C111 JJClrl L 


TTT 

L i ij 


1T1 1 PT1 PPT" 1 ATI 
111LU1 J LL L1U11 


PTKZU 


AflTtlplp i Tl fi TT i ti Vfl 1 711 

J_/CL1 UlL/lL/^ 11111111U1 V Cvl A/ Hi 


PTKNEG 


nQ rt 1 pi P TT P Q~P1 1 1 ATT 

JJCll llulL^ ll\jtj<XllL'll 


PTKVZS 


separated verbal prefix 


PTKANT 


particle, answer 


PTKA 


particle, adjectival or adverbial 


TRUNC 


truncated word 


$. 


punctuation, sentence delimiting 


$, 


punctuation, phrase delimiting 


$( 


punctuation, other 



Detailed guidelines on the use of the individual 



tags are available in (Thiclcn and Sailer, 1994) 



Table 1: The ELWIS tag set 



(5) Initial symbol biases are an additional set 
of biases used to define preferences for tag assign- 
ment given a particular equivalence class. Only a 
very few symbol biases were defined before evalua- 
tion of the first training runs, mainly to reflect bi- 
ases towards equivalence classes used in the class 
guesser. The majority of symbol biases were added 
to correct misguided biases chosen during the train- 
ing processes. 

(6) The two texts used for training the HMM 
were selected from the German data contained on 



the ECI's Multilingual CD-ROM (ECI, 1994): a 



200,000 and 2,000,000 word sample from Summer 
1992 issues of the German newspaper Frankfurter 
Rundschau. 

(7) The reference texts were also taken from 
the Frankfurter Rundschau, but do not overlap with 
the training texts. The reference texts amount to a 
total of approximately 20,000 running words, which 
were manually tagged and checked. 

(8) The current version of the implementation 
uses a straightforward tokenizer accepting one line 
per token. Training texts are pretokenized using an 
external tokenizer written in lex. 

2.1 Performance 

The best results of this implementation were ob- 
tained running 20 iterations of training over the 
200,000 word training text, using a total of 50 tran- 
sition and 17 symbol biases. With this configuration 
of training parameters, the resulting HMM assigned 
3.33 % incorrect tags when run on the reference texts 
and compared with the manually assigned tags. 

3 Comparison with other German 
models 

The main advantage of the Xerox tagger when com- 
pared with earlier implementations of HMM taggers 
is that it can be trained using untagged text. How- 
ever, the performance of the resulting HMM is very 
poor if no initial biases are used to help the training 
process find suitable parameters. 

For comparison , the evaluation procedure used to 



evaluate the implementation of the HMM tagger de- 

scribed in the preceding section was repeated with- 
out using any of the initial biases. The result was 
a poor performance of the resulting HMM with an 
error rate of 14.11 %. 

Choosing initial biases to help train a model is a 
subtle task in that it not only requires sound knowl- 
edge of the tag set used and the target language the 
model is aiming at, but it also requires a "feel" for 
how the initial biases may be modified during a given 



number of training iterations. It is also sometimes 
frustrating that the linguistic knowledge used to cre- 
ate the initial biases gets "optimized" or "trained" 
away in subsequent iterations of training. 

To overcome these disadvantages, hybrid tech- 
nologies have been developed that combine free text 
training methods with parameter estimation from 
pre-tagged texts. In such a setting, initial transi- 
tion and symbol biases are replaced by frequencies of 
tag sequences and tag instantiation from a relatively 
small pre-tagged corpus. The counted frequencies 
are taken as an approximation to the model's proba- 
bilities and get smoothed in a small number of train- 
ing iterations. 

In order to see what could be gained for a Ger- 
man language model by such a hybrid technology, 
I used extensions to the Xerox tagger developed at 
the University of Stuttgart that facilitate initializa- 
tion of an HMM with values obtained from a pre- 
tagged corpus (cf. ( |Schmid and Kempe, 1995| )). Ini- 
tial parameters were obtained by counting transition 
and symbol frequencies in a manually tagged 24,000 
word corpus taken from the newspaper Stuttgarter 
Zeitung, that was kindly made available by the Uni- 
versity of Stuttgart. These initial parameters were 
adjusted in a single training iteration using Xe- 
rox's Baum-Welch implementation for parameter 
re-estimation. The texts used for training and as 
a reference for evaluation were the same as the ones 
used in the implementation described in section |^. 
The resulting HMM had an error rate of 3.14 %. The 
superior performance of this model confirms results 



presented by (Briscoe et al., 1994), (Merialdo, 1994), 
and (Elworthy, 1994) for English: empirically ob- 
tained initial values for transition and output prob- 
abilities with a small number of training iterations 
lead to significantly better results than intuitively 
generated biases do. 

On the other end of the scale of parameter pro- 
duction for HMM POS disambiguators is the extrac- 
tion of parameters exclusively on the basis of (larger) 
pre-tagged text corpora, with no Baum-Welch re- 
estimation involved. Such an implementation is de- 



scribed in earlier work of the author (cf. ( Feldweg 
1993| ) and ( |Fcldwcg, 1995) )). For this model error 



rates of 3.16 % — 7.29 % (depending on the cover- 
age of the underlying lexicon) were reported. These 
results, however, are not directly comparable to the 
implementations described in this paper, since the 
tag sets employed differ to some extent. 

4 Assessment of ambiguity types 

In the preceding sections the evaluation of the model 
was purely quantitative. Performance was measured 



as the percent of mismatches between the output 
generated by the HMM and the tags assigned by 
manual tagging. Although this error rate is an ap- 
propriate measure for the performance of an HMM 
given a particular reference text, it says little about 
the amount of disambiguation done by the tagger, 
and nothing about the ambiguity types that were 
involved in the disambiguation process. 

The difficulty of disambiguation can be quantified 
by the ambiguity rate: the number of possible tag 
assignments divided by the number of words in a 
given text. The test text used to evaluate the Ger- 
man model described in section |^ has an ambiguity 
rate of 1.51, this is, the lexicon provides an average 
of 1.51 tags for each word in the text. 

In an effort to compare the German model with 
what is reported for other languages, English and 
French tagged texts were analysed. Both texts con- 
tained approximately 10, 000 words and were tagged 
using an English resp. French language model for the 
Xerox tagger. 

The tag set used to annotate the English text is a 
slightly modified version of the Brown tag set, con- 
sisting of a total of 72 tags. The ambiguity rate for 
this text is 1 .41. For the French text the t ag set 
described in ( Chanod and Tapanainen, 1994 ) with 
37 different tags is used. Here the ambiguity rate 
is 1.52. In terms of ambiguity rates, the English, 
French, and German texts are thus quite compara- 
ble. 

In order to compare the types of ambiguities that 
had to be resolved by the different language mod- 
els for the HMM tagger, relative frequency tables 
of equivalence classes were computed for each of 
the three texts. The top most frequent equivalence 
classes for the three languages are listed in table ^ 
together with their relative frequencies. 

If the table is viewed in terms of the major word 
classes of noun, verb, adjective, adverb, and closed- 
class forms, the following predominant ambiguity 
classes for English can be distinguished: 

• noun vs. verb (share, offer, plan), 

• adjective vs. noun (public, million, high), 

• closed-class vs. noun (a), 

• adjective vs. noun vs. verb (return, field), 

• closed-class vs. adverb (by, about, below), 

• and closed-class vs. adjective vs. adverb (round, 
next), 

For French the major ambiguity types are: 



• noun vs. verb (affaire, bout, place), 

• adjective vs. noun (demi, moyen, responsable, 

• adjective vs. verb (applique, devenu, fabrique), 

• closed-class vs. adjective (numeral un). 

The elements of the most frequent ambiguity types 
for German, however, belong to the same major 
word classes, with only a few exception such as: 

• closed-class vs. adjective (numeral einen, 
einer), 

• and verb vs. adjective (fehlgeschlagen, bekannt), 

with the latter reflecting a subtle distinction in the 
German tag set (VPP vs. ADJD: participle as mod- 
ifier vs. non-attributive adjectives). 

The comparison of the most frequent ambiguity 
types shows a significant difference between the Ger- 
man model on the one hand and the English and 
French models on the other. In German most of 
the effort is going into subclassification within ma- 
jor word classes, while in English and French a good 
deal of disambiguation work is devoted to separate 
major word classes. 

5 Assessment of error types 

The differences in ambiguity types of the models also 
have effects on the types of errors produced by the 
German model. Again, errors mainly affect the as- 
signment of words to subclasses within one major 
word class. 

Table |^ shows the most common errors produced 
by the German model. The entries are sorted by 
decreasing frequencies relative to the total number 
of mismatches between the manually and automati- 
cally tagged texts. The first column gives the rela- 
tive frequenciy and the second column lists the tag 
chosen by the German HMM tagger. In the second 
column, the number following the slash indicates the 
number of elements in the equivalence class from 
which the model had to choose. A missing num- 
ber indicates that there was only one choice in the 
lexicon. The third column show the "correct" tag, 
as chosen by the human tagger. 

The most common (accumulated) error is the con- 
fusion of proper nouns and common nouns — a re- 
sult of the fact that both proper nouns and common 
nouns are both capitalized in German. The fourth 
line of table |^ represents a special case of this error 
for which the HMM model can not be held respon- 
sible: no ambiguity was indicated in the lexicon, so 
the model had nothing to choose from. This oc- 
curs most frequently when common nouns are used 





English 




French 




German 


f(ec) 


elements of equiv. class 


f(ec) 


elements of equiv. class 


f(ec) 


elements of equiv. class 


.0701 


NN VB 


.0862 


DET-PL PREP 


.0772 


ART PROS PRELS 


.0441 


VBD VBN 


.0678 


DET-SG PC 


.0265 


PTKVZS APPR 


.0357 


JJ NN 


.0263 


NOUN-SG VERB-P1P2 


.0255 


NE NN 








VERB-P3SG 






.0301 


NNS VBZ 


.0233 


ADJ-SG NOUN-SG 


.0252 


VINF VFIN 


.Uzz4 


A T 1 TVTT5 


.Ul 1 4 


A T\ T Cir 1 DAD Gfi 


.una 


A TYW If C\ TVT 

AJJV IvUIn 


.0118 


JJ NN VB 


.0158 


DET-PL PC 


.0117 


ART PROS PROAT 












CARD 


.0111 


IN RB 


.0125 


PC PREP 


.0116 


VPP ADJD 


.0097 


PPO PPS 


.0119 


DET-SG NUM PRON 


.0095 


VPP ADJD VFIN 


.0097 


IN JJ RB 


.0118 


DET-SG PREP 


.0089 


PROS PROAT 


.0094 


CS DT WPS 


.0109 


ADJ-PL NOUN-PL 


.0086 


PTKVZS APPO APPR 












APZR 



f(ec) = relative frequency of equivalence class 



Tabic 2: Elements of 10 most frequent ambiguous equivalence classes for English, German and French 



Rel.Freq 


HMM 


Human 


0.0900 


VINF/2 


VFIN 


0.0790 


NN/2 


NE 


0.0648 


NE/2 


NN 


0.0521 


NN 


NE 


0.0332 


NE/7 


NN 


0.0316 


VPP/3 


VFIN 


0.0269 


VPP/3 


ADJD 


0.0269 


ADV/2 


KON 


0.0237 


APPO/6 


APPR 


0.0205 


PROS/3 


ART 


0.0205 


PROS/2 


PWS 


0.0158 


P WAV/4 


KOKOM 


0.0158 


PRELS/3 


PROS 


0.0158 


ART/3 


PROS 


0.0158 


ART/3 


PRELS 


0.0142 


VFIN/2 


VINF 


0.0126 


VPP/2 


ADJD 


0.0126 


VINF/3 


VFIN 


0.0126 


KOUS/2 


APPR 


0.0126 


KON/4 


KOKOM 



Table 3: 20 most common error types of German 
HMM 



as proper nouns (e.g. in die gehobene Mittelklasse 
plaziert Renault ab 6. Marz den Safrane) , where one 
would not expect to add a tag "proper noun" for 
every noun in the lexicon. 

The second most frequent error type involves con- 
fusion of infinitives and 1st and 3rd pers. pi. fi- 
nite present tense forms. These are homographs in 
German that are notoriously hard to disambiguate 
within a narrow context. 



The difficulty of distinguishing between non- 
attributive, adjectival usage of participles (i.e. er ist 
geladen) and participles proper (i.e. er hat den Wa- 
gen geladen) was mentioned in the preceding sec- 
tion. In addition a number of these forms may also 
be used as finite verbs (i.e. erhalten, gehort), and 
this is a further source of errors. 

Almost all of the remaining errors are misassign- 
ments within closed classes, including well-known 
errors due to long distance phenomena, such as those 
resulting from the confusion of relative pronouns, 
demonstrative pronouns and articles in sentences 
like: die einmal fur die Buchproduktion erfaflten 
Texte or: doch der wollte nicht, das falle auf. 

6 Conclusion 

Despite the hypothesis that the free word order 
of German leads to poor performance of low or- 
der HMM taggers when compared with a language 
like English, the overall results for German are very 
much along the lines of comparable implementations 
for English, if not better. It can be argued that the 
disadvantage of free word order for HMM taggers is 
compensated for by richer morphology and the addi- 
tional disambiguation cue of having upper and lower 
case initial letters to distinguish POS membership. 
The latter, however, greatly hinders the recognition 
of proper nouns, the most common type of error, 
responsible for approximately 20 % of the model's 
3.33 % mistakes. 

It is important to notice that the types of dis- 
ambiguation carried out by the tagger for German 
are significantly different from the disambiguation 
work for English and French. While in English and 
French a fair number of disambiguations involve sep- 



arating major POS classes such as verb, noun, and 
adjective, most of the work performed in the German 
model involves disambiguation between subclasses of 
one main category, such as finite vs. infinitive verb, 
noun vs. proper noun, different sub-categories of pro- 
nouns, etc. 

This finding has consequences for the COMPASS 
project, where POS disambiguation is employed as 
one means of disambiguating word senses to facil- 
itate precise dictionary look-up. While this tech- 
nique helps to confine word senses for English and 
French, it is of little help for word sense disambigua- 
tion in German. 

However, the German model was useful for the 
project because a tagged reference corpus was re- 
quired for lexicographic work in order to adapt ex- 
isting bilingual dictionaries. The tagger was used 
to annotate all of the 50 million word German cor- 
pora contained on the ECI Multilingual Corpus 1 
CD-ROM. 

Acknowledgements 

I would like to thank Helmut Schmid of the Univer- 
sity of Stuttgart for providing extensions of param- 
eter initialization for the Xerox Tagger and Jean- 
Pierre Chanod and Lauri Karttunen of the Rank Xe- 
rox Research Laboratory Grenoble for making avail- 
able the English and French tagged texts and lexi- 
cons. I would also like to acknowledge valuable ad- 
vice from Tracy Holloway King and Steven Abney, 
who commented on earlier versions of this paper. 

This work has been supported in part by the Min- 
istry for Science and Research of the Land Badcn- 
Wurttemberg under the project "Corpus Based De- 
velopment of Lexical Knowledge Bases (ELWIS)" 
and by the Commission of the European Community 
under the LRE project "Adapting Bilingual Dictio- 
naries for Online Assistance (COMPASS, LRE 62- 
080)". 

References 

[Briscoe et al.1994] Ted Briscoe, Greg Grefenstette, 
Llufs Padro, and Iskander Serail. 1994. Hybrid 
techniques for training HMM part-of-speech tag- 
gers. Acquilcx II working paper 45. 

[CELEX1993] CELEX. 1993. The CELEX Lexical 
Database. Dutch, English, German. Max-Planck- 
Institute for Psycholinguistics, Centre for Lexical 
Information, Nijmegen. CD-ROM. 

[Chanod and Tapanainenl994] Jean-Pierre Chanod 
and Pasi Tapanaincn. 1994. Statistical and 



constraint-based taggers for French. Technical 
Report MLTT - 016, Rank Xerox Research Cen- 
tre, Grenoble Laboratory, Grenoble. 

[Cutting et al.1992] Doung Cutting, Julian Kupiec, 
Jan Pedersen, and Penelope Sibun. 1992. A prac- 
tical part-of-speech tagger. In Proceedings of the 
Third Conference on Applied Natural Language 
Processing, Trento. 

[ECI1994] ECI. 1994. Multilingual Corpus 1. Asso- 
ciation for Computational Linguistics, European 
Corpus Intitiative. CD-ROM. 

[Elworthyl994] David Elworthy. 1994. Docs Baum- 
Welch re-estimation help taggers? In Proceedings 
of the Fourth Conference on Applied Natural Lan- 
guage Processing, Stuttgart, pages 53-58. 

[Feldwegl993] Helmut Feldweg. 1993. Stochastische 
Wortartcndisambiguicrung fur das Deutsche: Un- 
tersuchungen mit dem robusten System LIKELY. 
Technical report, Universitat Tubingen, Seminar 
fur Sprachwissenschaft. 

[Feldwegl995] Helmut Feldweg. 1995. Stochastis- 
che Wortartendisambiguicrung des Dcutschen. In 
Lexikon & Text, pages 241-254, Tubingen. Max 
Niemeyer. forthcoming. 

[Kupiec and Wilkensl994] Julian Kupiec and Mike 
Wilkcns. 1994. The DDS tagger guide version 
1.1. Xerox Palo Alto Research Center, unpub- 
lished manuscript. 

[Merialdol994] Bernard Merialdo. 1994. Tagging 
English text with a probabilistic model. Compu- 
tational Linguistics, 20(2):155-171. 

[Schmid and Kempel995] Helmut Schmid and Andre 
Kcmpc. 1995. Tagging von Korpora mit HMM, 
Entschcidungsbaumen und neuronalen Netzen. In 
Helmut Feldweg and Erhard Hinrichs, editors, 
Lexikon & Text, Tubingen. Max Niemeyer. forth- 
coming. 

[Thiclcn and Sailerl994] 

Christine Thielen and Manfred Sailer. 1994. Ein 
Tagset furs Deutsche. Richtlinien fur die manucllc 
Wortarten-Annotierung von Textkorpora. Semi- 
nar fur Sprachwissenschaft, Universitat Tubingen, 
unpublished Manuscript. 

[Thielen and Schillerl995] Christine 

Thielen and Anne Schiller. 1995. Ein kleines und 
erweitertes Tagset furs Deutsche. In Helmut Feld- 
weg and Erhard Hinrichs, editors, Lexikon & Text, 
pages 215-226. Max Niemeyer, Tubingen, forth- 
coming. 

[Thielenl994] Christine Thielen. 1994. Ein 
Tagset fur die Wortartenklassifizierung des 



Deutschen. In Harald Trost, editor, KONVENS 
'94- Ostcrreichische Gesellschaft fur Artificial In- 
telligence, Wien. 

[Wothke et al.1993] Klaus Wothke, Ilona Weck-Ulm, 
Johannes Hcincckc, Oliver Mcrtincit, and Thomas 
Pachunke. 1993. Statistically based automatic 
tagging of German text corpora with parts-of- 
speech — some experiments. Technical report, 
IBM Germany, Heidelberg Scientific Center, Hei- 
delberg. 



